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Abstract 

In this paper, we give necessary and sufficient conditions for low-density parity-check (LDPC) codes 
with column-weight three to correct three errors when decoded using hard-decision message-passing decoding. 
Additionally, we give necessary and sufficient conditions for column-weight-four codes to correct three errors in 
four iterations of hard-decision message-passing decoding. We then give a construction technique which results 
in codes satisfying these conditions. We also provide numerical assessment of code performance via simulation 
results. 

I. Introduction 

First introduced by Gallager [[Tl, LDPC codes have been the focus of intense research in the past 
decade and many of their properties are now well-understood. The iterative decoding algorithms for 
LDPC codes have been analyzed in detail, and asymptotic performance results have been derived 
However, estimation of frame-error-rate (FER) for iterative decoding of finite-length LDPC codes is still 
an unsolved problem. A special case of interest is the performance of iterative decoding at high signal- 
to-noise ratio (SNR). At high SNRs, a sudden degradation in the performance of iterative decoders has 
been observed Il3l,[l4]|. This abrupt change manifested in the FER curve is termed as an "error-floor." 

The error-floor problem is well-understood for iterative decoding over the binary erasure channel 
(BEC) [|5l. Combinatorial structures called "stopping sets" were used to characterize the FER for iterative 
decoding of LDPC codes over the BEC. It was established that decoding failure occurs whenever all 
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the variables belonging to stopping sets are erased. Tian et al. f6l| used this fact to construct irregular 
LDPC codes which avoid small stopping sets thus improving the guaranteed erasure recovery capability 
of codes under iterative decoding, and hence improving the error-floors. As in the case of BEC, a 
strong connection has been found between the existence of low-weight uncorrectable error patterns and 
error-floors for additive white Gaussian noise (AWGN) channels and binary symmetric channels (BSC) 
(see [4J and 0). Hence, studying the guaranteed error correction capability of codes under iterative 
decoding is important in the context of characterization and improvement of the performance of iterative 
decoding strategies. 

In the past, guaranteed error correction has been approached from the perspective of the decoding 
algorithm as well as from the perspective of code construction. Sipser and Spielman [8J used expansion 
arguments to derive sufficient conditions for the parallel bit-flipping algorithm to correct a fraction 
of errors in codes with column-weight greater than four. Burshtein [|91| proved that for large enough 
lengths, almost all codes with column-weights greater than or equal to four can correct a certain fraction 
of errors under the bit-flipping algorithm. Burshtein and Miller [fTOll derived the sufficient conditions 
for message-passing decoding to correct a fraction of errors for codes of column-weight greater than 
five. However, these proofs were not constructive, i.e., no explicit code construction which satisfied the 
sufficient conditions was provided. Moreover, the code-lengths required to guarantee the correction of 
a small number of errors (say 3) is very high . Also, these arguments cannot be extended for message- 
passing decoding of codes with column-weight three or four. 

In order to construct codes with good error correcting properties under iterative decoding, progressive 
edge growth (PEG) [fTTl and constructions based on finite geometries [[T2ll have been used. However, 
codes constructed from finite geometries typically have very high column-weight. Although, it has 
been proved that minimum distance grows at least linearly for codes constructed using PEG, no results 
proving guaranteed error correction under iterative decoding exist for these codes. 

In this work, we derive necessary and sufficient conditions for the correction of three errors in a 
column-weight-three code under the hard-decision message-passing algorithm. We provide a modified 
PEG construction which yields codes with such an error-correction capability. Also, we derive the 
necessary and sufficient conditions for the correction of three errors in four iterations for the case of 
codes with column-weight four. Again, we provide a modified PEG construction which yields codes 
with such error-correction capability. 

The remainder of the paper is organized as follows: We establish the preliminaries of the work in 



SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY, OCTOBER 2008 



3 



Section |lll The necessary and sufficient conditions for the correction of three errors in column-weight- 
three codes are derived in Section Unl The case of column-weight-four codes is dealt with in Section 
HVl In Section |Vl we describe a technique to construct codes satisfying the conditions of the theorems 
and provide numerical results. We conclude with a few remarks in Section |VIl 

II. Preliminaries 

In this section, we first describe the Tanner graph representation of LDPC codes. Then, we establish 
the notation that will be used throughout this paper. Finally, we describe the hard-decision message- 
passing algorithm that will be used for decoding. 

A. Notation 

The Tanner graph of an LDPC code, Q(y,C), is a bipartite graph with two sets of nodes: V, the 
variable (bit) nodes and C, the check (constraint) nodes. Every edge e in the bipartite graph is associated 
with a variable node v and a check node c. The check nodes (variable nodes, respectively) connected 
to a variable node (check node, respectively) are referred to as its neighbors. The degree of a node is 
the number of its neighbors. In a (7, p)-regular LDPC code, each variable node has degree 7 and each 
check node has degree p. The girth g is the length of the shortest cycle in Q. Let S C V such that 
l^l = y. If for all choices of S, there are at least z neighbors of S in C, then we say that the y ^ z 
condition is satisfied. In this paper, • represents a variable node, □ represents an even-degree check 
node and ■ represents an odd-degree check node. 

B. Hard-Decision Decoding Algorithm 

Let r = [r(l), r(2), . . . , r(ri)], a binary n-tuple, be the input to the message-passing decoder. Let 
V E V he a variable node with r(v) as its corresponding bit and c G C be a check node neighboring 
V. Let ujj{v,c) denote the message that v sends to c in the first half of the j*'' iteration and iJj{c,v) 
denote the message that c sends to v in the second half of the j*^ iteration 

Additionally, let ujj{v, :) be the set of all messages from a variable v to all its neighboring checks in 
the first half of the iteration. Let ujj{v, : \c) be the set of all messages that a variable node v sends 
to all its neighboring checks except c in the first half of the j*^ iteration. Let ujj{:, v) be the set of all 
messages received by v from all its neighboring in the second half of the j*'' iteration. Let cJj(: \c, v) 
be the set of all messages received by v from all its neighboring check nodes except c in the second 
half of the j*'' iteration. ujj{c, :), ujj{c, : \v), ujj{:, c) and ujj{: \v, c) are defined similarly. 
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The Gallager algorithms ^ can be defined as follows: The forward messages, ujj{v, c) (from variables 
to checks), are defined as 

r{v), if j = 1 

Uj{v,c) = { m, if \{c' : c' ^ c,ZJj_i{c' ,v) = m}\ > b.,j (1) 
r{v), otherwise 

where \{c' : c' ^ c,iJj-i{c' ,v) = m}\ refers to the total number of messages which are of the value 
m e {0, 1}. The backward messages, iJj(c,v) (from checks to variables), are defined as 

ujj{c,v) = ^ nij mod 2 (2) 

\mjeiLjj{c,:\v) j 

At the end of each iteration, an estimate of each variable node is made based on the incoming 
messages and possibly the received value. The decoder is run until a valid codeword is found or until 
a maximum number of iterations, say D, is reached, whichever is earlier. 

In Eqn. ([T]), 6^ j is a threshold which is generally a function of the iteration number, j, and the degree 
of the variable v. In this paper, we use 6^ ^ = 2 for all v and j for decoding column- weight-three codes. 
For column- weight-four codes, we use = 3 for all v when 1 < j < 3 and 6„j = 2 for all v when 
J >4. 

Remark: We note that Eqns. [Hand [21 then correspond to the Gallager-B algorithm [[ij. For the Gallager- 
A algorithm [IJ, h^ j =7^ — 1, for all j, where 7u is degree of variable node v. 



A Note on the Decision Rule: Different rules to estimate a variable node after each iteration are 
available, and it is likely that changing the rule after certain number of iterations may be beneficial. 
However, the analysis of such scenarios is beyond the scope of this paper. Throughout the paper, we 
use the following decision rule: if all incoming messages to a variable node from neighboring checks 
are equal, set the variable node to that value; else set it to its received value. 

C. Trapping Sets of the Hard-Decision Decoder 

We discuss briefly the concept of trapping sets. Consider an LDPC code of length n. Let r be the 
binary vector which is the input to the hard-decision decoder. For output symmetric channels, without 
loss of generality, we can assume that the all-zero-codeword is transmitted. We make this assumption 
throughout this paper. The support of a vector r denoted by 5(r) is defined as the set of all positions 
i where r{i) ^ 0. For each I, I < I < D, let x' be the codeword estimate of the decoder at the end of 
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the / iteration. A variable node v is said to be eventually correct if there exists a positive integer If. 
such that for all I > Ic, v does not belong to 5(x'). 

Definition 1: [4J A decoding failure is said to have occurred if there does not exist / < D such that 
5(xO = 

Definition 2: dH Let T(r) denote the set of variable nodes that are not eventually correct. If T(r) is 
not empty, let a = |T(r)| and b be the number of odd-degree check nodes in the subgraph induced by 
T(r). We say that T(r) is an (a, 6) trapping set. 

Definition 3: Let T he a trapping set and let R(T) = {r : T(r) = T}. The critical number ^ of 
trapping set T is the minimum number of variable nodes that have to be initially in error for the decoder 
to end up in the trapping set T. That is, ^ = minreR(T) |'5(r)|. 

Definition 4: [4] Let T he a trapping set. If T(r) = T, then S{r) is a failure set of T. 

Definition 5: For transmission over the BSC, r is a fixed point of the decoding algorithm if S{r) = 
5(xO for all /. 

It follows that for transmission over the BSC, if r is a fixed point, then T(r) = S{r) is a trapping set. 
Now, we have the following theorem which provides the sufficient condition for a set of variables to 
be a trapping set: 

Theorem 1: [fT3l Let G(y,C) he the Tanner graph of a column-weight-three code. Let T C V,he 
a set consisting of v variable nodes with induced subgraph I. Let the checks in I he partitioned into 
two disjoint subsets, namely, O consisting of checks with odd degree and £ consisting of checks with 
even degree. If (a) every variable node in J is connected to at least two checks in £ and at most one 
check in O and (b) no two checks of O are connected to the same variable node outside X, then T is 
a trapping set. 

Proof: See ti3J. ■ 

III. Column-Weight-Three Codes 

In this section, we establish necessary and sufficient conditions for a column-weight-three code to 
correct three errors. We first illustrate three trapping sets and show that the critical number of these 
trapping sets is three thereby providing necessary conditions to correct three errors. We then prove 
that avoiding structures isomorphic to these trapping sets in the Tanner graph is sufficient to guarantee 
correction of three errors. 

Fig. [H shows three subgraphs induced by different numbers of variable nodes. Let us assume that in 
all these induced graphs, no two odd degree checks are connected to the same variable node outside 
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the graph. By the conditions of Theorem [B all these induced subgraphs are trapping sets. Fig. |l(a) 



IS 



a (3, 3) trapping set, Fig. |l(b)| is a (5, 3) trapping set and Fig. |l(c)| is a (8, 0) trapping set. Note that 
a (3, 3) trapping set is isomorphic to a six-cycle and the (8, 0) trapping set is a codeword of weight 
eight. We now have the following result: 






(a) 



(b) 



(c) 



Fig. 1. Examples of trapping sets with critical number three: [(a)] a (3, 3) trapping set; |(b)| a (5, 3) trapping set; and |(c)| an (8, 0) trapping 
set. 

Lemma 1: The critical number for a (3, 3) trapping set which is also a fixed point is at most three. 
There exist (5,3) and (8,0) trapping sets with critical number three and no (5,3) or (8,0) trapping sets 
with critical number less than three. 

Proof: The proof for (3, 3) case is trivial. We prove the lemma for the case of (5, 3) trapping sets 





(a) 



(b) 





(c) 



(d) 



Fig. 2. Illustration of message passing for a (5, 3) trapping set: |(a)| variable to check messages in round one; |(b)| check to variable 
messages in round one; |(c)| variable to check messages in round two; and |(d)| check to variable messages in round two. Arrow-heads 
indicate the messages with value 1. 



and omit the proof for (8, 0) trapping sets. 
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Consider the (5, 3) trapping set shown in Fig. [2l Let := {vl, vl,vl} be the set of variables which 
are initially in error. Let := {c\, cl, . . . , Cg} and V'^ := {vj, t>|}. Also, assume that no variable node 
in V\{V^ U V"^), has two or more neighbors in C^. In the first iteration, we have: 

1 ifveV^ 

ujiiv,c) = { (3) 

otherwise 

1 if ceC\ v^V^ 
ZJiicv) = { ^ (4) 

otherwise 

Consequently, all variable nodes in V"^ are decoded incorrectly at the end of the first iteration. In the 
second iteration: 

' 1 ifveV^ 

UJ2{V,C) = { (5) 

otherwise 

, 1 if ceC'\{clcl 4} , viV^ 

^2\C,V) = <^ (6) 

otherwise 

and all variable nodes in V'^ are decoded incorrectly. Continuing in this fashion, co3{v, c) = toi{v, c) and 
cl;3(c, f) = EJi(c, f). That is, the messages being passed in the Tanner graph would repeat after every 
two iterations. Hence, three variable nodes in error initially can lead to a decoder failure and therefore, 
this (5, 3) trapping set has critical number equal to three. ■ 
Theorem 2: To guarantee that three errors in a column- weight-three LDPC code can be corrected by 
the Gallager-A algorithm, it is necessary to avoid (3, 3), (5, 3) and (8, 0) trapping sets in its Tanner 
graph. 

Proof: Follows from the discussion above. ■ 
We now state and prove the main theorem. 

Theorem 3: If the Tanner graph of a column-weight-three LDPC codes has girth eight and does not 
contain a subgraph isomorphic to a (5, 3) trapping set or a subgraph isomorphic to an (8, 0) trapping 
set, then any three errors can be corrected using the Gallager-A algorithm. 

Proof: Let := {v\,v\,v\} be the three erroneous variables and be the set of the checks 
connected to the variables in . In a column- weight-three code (free of cycles of length four) the 
variables in can induce only one of the five subgraphs given in Fig. [3l In each case, uJi{v, :) = {!} 
if V E and is otherwise. The proof proceeds by examining these subgraphs one at a time and 
proving the correction of the three erroneous variables in each case. 
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r' r' r' r' r' r' I l l I 1 1 I l l l l l l l 

C| C3 C4 C5 q (-^ q 




(a) (b) (c) 



c; c\ c\ c\ c\ c\ c\ c\ c\ c\ c\ c\ c\ c\ cl c\ 

(d) (e) 
Fig. 3. All the possible subgraphs that can be induced by three variable nodes in a column-weight-three code 

Subgraph 1: Since the girth of the code is eight, it has no six cycles. Hence, the configuration in 



Fig. |3(a)| is not possible. 

Subgraph 2: The variables in induce the subgraph shown in Fig. |3(b)[ At the end of the first 
iteration: 

' 1 a cec\ V iv^ 



^i[c,v) = { (7) 
otherwise 

There cannot exist a variable node which is connected to two or more checks in the set without 
introducing either a six-cycle or a subgraph isomorphic to (5, 3) trapping set. At the end of first iteration, 
ZJi(:,v) = {0} for all v E V^. Furthermore, there exists no v ^ for which ZJi(:,v) = {1}. Hence, if 
a decision is made after the first iteration, a valid codeword is found and the decoder is successful. 

Subgraph 3: The variables in induce the subgraph shown in Fig. |3(c)[ At the end of the first 
iteration: 

1 if cgci\{4, 4}, v^v^ 

uJi{c,v)=l 1 if ce {clcl}, V eV^ (8) 
otherwise 

For no f ^ V^, uJi{:,v) = {1} as this would introduce a four-cycle or a six-cycle in the graph. For any 
V ^ V^, uj2{v, c) = 1 only if Ui(: \c, v) = {1}. This implies that v has two checks in C^\{cl, cl}. Let 
V'^ be the set of such variables. We have the following lemma: 
Lemma 2: There can be at most one variable in V"^. 

Proof: Suppose = 2. Specifically, assume V"^ = {^1,^2}- The proof is similar for > 2. 
First note that for any v E V^, v cannot be connected to c\ as it would create a six-cycle. Next, 
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let Cl :— {c\,cl} and C2 := {cgjCy}. Then, v cannot have both checks in either Cl or C2 as this 
would cause a four-cycle. Hence, v has one check in C| and one check in C^. Assume without loss of 
generality that vj is connected to c\ and Cg. Then, t>| cannot be connected to c\ and c], as this would 
form a six-cycle. ^1 cannot be connected to and as it would create a (5, 3) trapping set. Hence, 
\V^\<2. U 
Let vj e V"^ be connected to c\, cl and an additional check cf. In the second iteration: 

1 if t;G {vlvl}, c^{clcl} 



UJ2{V, c) = < 



(9) 



u;2{c,v) = < 



(10) 



I ifv 
1 ifv 

otherwise 

1 if c e C^\{cl}, v^V^ 
1 if c e {4,4,4}, v^vl 
1 if c = 4, V 7^ 
otherwise 

We have the following lemma: 

Lemma 3: There cannot exist any variable v ^ F ^ (J such that it receives two or more incorrect 
messages at the end of the second iteration. 

Proof: Suppose there existed a variable v such that it received two incorrect messages in the second 
iteration. Then, it would be connected to two checks in the set C^|J{4}- This is not possible as it 
would introduce a four-cycle, six-cycle or a (5, 3) trapping set {e.g. if v is connected to c\ and c\, it 
would form a (5, 3) trapping set). ■ 

Thus, in the third iteration: 



1 if V G {4,4}' c i {4,4} 



1 liv 



^1, 



(11) 



otherwise 



1 if c e (4, 4, 4, C7}, ^ i {^1, ^^3} 



1 ifc 



'1, 



(12) 



otherwise 



At the end of the third iteration, uj2,{:, v) — {0} for all v e V^. Also, we have the following lemma: 
Lemma 4: There exists no v such that 'iJJ2,{:,v) — {!}. 
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Proof: Suppose there exists v such that uj^{:,v) = {1}. Then, v is connected to three checks in 
the set {c\,cl,cl,c],cl}. This implies that uJ2i-,v) = {!}. However, from Lemma [3] it is evident that 
no such V exists. ■ 
Hence, if a decision is made after the third iteration, a valid codeword is found and the decoder is 
successful. 

Subgraph 4: The variables in induce the subgraph shown in Fig. |3(d)[ At the end of the first 
iteration: 

1 if c e C^\{cl}, v^V^ 
uji{c,v) = { 1 if c = cl, V e {vl,vl} (13) 
otherwise 

For no V e V\V^, uj2i-,v) = {1}. For any v e VXV^, uj2{v,c) = 1 only if cJi(: \c,v) = {!}. Let V'^ 
be the set of all such variables. We have the following lemma: 

Lemma 5: (i) V"^ has at most four variables, and (ii) No two variables in V"^ can share a check in 

C\C\ 



Sketch of the Proof: There exists no variable which is connected to two checks from the set {c\,c\,c\, 



45 



as it would introduce a four-cycle or a six-cycle. However, a variable node can be connected to one 
check from {c\, c\, c\, c\, c\} and to one check from {cg, c\, c\}. There can be at most four such variable 
nodes. When four such variable nodes exist, none are connected to c\. Also, these four variable nodes 
cannot share checks outside the set C^\{c\}. 

Let these four variable nodes be labeled ff, and vl and their third checks c\, c\, c\ and c\, 

respectively. Let := {c^, c|, C3, c|}. Hence, in the second iteration: 



1 ifve {vl,vl}, c^cl 

1 if V ceC^ 

otherwise 



(14) 



UJ2[C,V) 



1 if c G {cl, c\, cl, cl}, V ^ {vl, vl} 
1 if ceC\ V ^V^ 
otherwise 



(15) 



At the end of the second iteration iJ2{:,v) = {0} for all v eV^. Moreover, for nov ^ V-^, uj2{:, v) = {!}. 
So, if a decision is made after the second iteration, a valid codeword is reached and the decoder is 
successful. 
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Subgraph 5: The variables in induce the subgraph shown in Fig. |3(e)[ At the end of the first 
Iteration: 

f 1 if ce C\ V ^V^ 

(16) 



1 if ce C\ V ^V^ 
otherwise 

If there exists no variable v E V\V^ such that iJi(:,v) = {1}, a valid codeword is reached after the 
first iteration. Suppose this is not the case. Let V"^ be the set of variables which receive two or more 
incorrect messages. Then, we have the following lemma: 

Lemma 6: (i) There exists one variable vl E such that ZJi(:,vf) = {!}, and (ii) V'^ has at most 
three variables which receive two incorrect messages at the end of the first iteration. Furthermore, they 
cannot share a check in C\C^. 

Proof: We omit the proof of Part (ii) as it is straightforward. Part (i) is proved as follows: If there 
existed no variable, vj, such that uji{:,vj) = {1}, then the decoder would converge in one iteration. 
Next, suppose E V"^ such that Ui(:,f^) = cJi(:,f|) = {1}. Without loss of generality, let vj be 

connected to c\,cl and c]. Then, vf would share two checks in the set C^. It is thennot possible to 
connect t>| without introducing a six-cycle or a (5, 3) trapping set {e.g., if vf is connected to cl, cl and 
Cg, then it would introduce a (5, 3) trapping set). ■ 

Let the third checks connected to vl and f| be cf, and c^, respectively and let := {cl,C2,cl} 
In the second iteration: 



UJ2iv,c) 



1 if V = vf 

lifvE V^\{vl}, c E C 
otherwise 



(17) 



1 if c G {cj,4,4}, V ^ vl 
U2{c,v) = < 1 if ceC^,v^V^ (18) 
otherwise 

There cannot exist a variable node which is connected to one check from and to one check from 
{vl, vl, Vj}. Also, there cannot be a variable node which is connected to all three checks in the set 
as this would introduce a graph isomorphic to the (8, 0) trapping set. However, there can be at most 
two variable nodes which receive two incorrect messages from the checks in C^, say vf and f f . Let the 
third checks connected to them be cf and c^, respectively. Let := and := {cfjCg}. At 

the end of the second iteration, variables vl, v\ and ^3 receive one incorrect message each. Variables 
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in the set receive two incorrect messages each. Therefore, in the third iteration, we have: 

1 ifveV\c^ {cl,ci,4} 
uj3{v,c) = { 1 if V ceC^ (19) 

otherwise 

1 if ce C^\{c\, eld}, v^V^ 
1 if ceC\ V ^V^ 

At the end of the third iteration, cU3(:,t>) = {1} for all v E V^. Furthermore, for no f ^ V^, cJ3(:,t>) = 
{1}. So, if a decision is made after the third iteration, a valid codeword is reached and the decoder is 
successful. ■ 

IV. Column-Weight-Four Codes 

In this section, we derive necessary and sufficient conditions for the correction of three errors in 
column-weight-four codes in four iterations of iterative decoding. This result is inspired by the analysis 
of error events in high-rate codes with column- weight four. In simulations, it was found that received 
vectors which did not converge to a valid codeword in the first 4 to 5 iterations did not converge 
thenceforth. Hence, it is desirable to devise codes and decoding strategies in which vectors having a 
small number of errors converged rapidly to a codeword. To this end, it was found that a hybrid decoding 
strategy could correct three errors in four iterations if certain conditions are satisfied by the code. This 
result is summarized as follows: 

Theorem 4: An LDPC code with column-weight four and girth six can correct three errors in four 
iterations of message-passing decoding if and only if the conditions, 4 — ^> 11, 5 — 12, 6 — > 14, 7 16 
and 8 18 are satisfied. 

Remark: It is worth noting that if a graph of girth six satisfies the 4^11 condition, then it satisfies 
the 5 ^ 12 condition as well. However, the addition of this extra constraint aids in the proof of the 
theorem. 

Proof: First, we prove the sufficiency of the conditions of Theorem HI 

Let := {vl, vl,vl} be the three erroneous variables. Let be the set of checks that are connected 
to the variables in V'^. The variables in can induce only one of the five subgraphs shown in Fig. HI 
We prove that in each case, the decoding algorithm converges to the correct codeword in four iterations. 

Subgraph 1: The variables in induce the subgraph shown in Fig. |4(a)[ At the end of the first 



iteration, uji{:,v) = {0} for all v G V^. Moreover, no variable receives four incorrect messages after 
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Vi V2 V3 

.-A-^ ^-A^ 

c' c' c' c' c' c' c' c' c' c' c' c' 

^2 L4 L5 L-g L|Q L.JI C[2 



(a) 



^' ^1 ^' ^' ^' ^' ^' ^1 ^' ^' ^' „i „i „i „i „i „i „i ?T ^ ?i 



Cj C2 C4 C5 Cy Cq C|fi C] 



9 ''lO ''ll 



C[ C2 C3 C4 C5 Q Cy Cg Cg Cjo 



(b) 



vl 



"1 ^-2 ''3 '-4 ^5 ^-6 ^7 



(c) 




Cy Cj C3 C4 C5 Cy Cfj Cg 



(d) (e) 
Fig. 4. All the possible subgraphs that can be induced by three variable nodes in a column-weight-four code 



the first iteration as the existence of such a variable node would create a four-cycle. If a decision is 
made after the first iteration, the decoder is successful. 

Subgraph 2: The variables in V'^ induce the subgraph shown in Fig. |4(b)[ At the end of the first 
iteration: 

1 if c G C^\{cl}, v^V^ 
uji{c,v) = l 1 if c = cl, V e {vl,vl} (21) 
otherwise 

For no f G V\V^, uji{:, v) = {1} as it would introduce a four-cycle. For any v ^ V^, uj2{v, c) = 1 only 
if cJi(: \c,v) = {1}. This implies that v is connected to three checks in C^\{cl}. Let V"^ denote the 
set of such variables. We have the following lemma: 

Lemma 7: There can be at most three variables in V'^. Furthermore, no two variable nodes in V'^ 
share any check in the set C\C^. 

Proof: Let V"^ = {vf, t>|, v^, Then the set of variable nodes [J V"^ has at most 15 neighboring 
checks. This violates the 7 ^ 16 condition. Hence, V'^ can have at most three variables. Next, let 
G V^. Suppose they share a fourth check c. Since ^3 can share at most two checks with vf and 
t>2, assume that c\q and c\i are not neighbors of vf, The neighbors of the variable nodes in the 
set {vl, f 2, vf, f|} all belong to the set {c\, ■ . ■ ,cl} IJ{ci} which has cardinality 10, thus violating the 
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4^11 condition. ■ 
Let the fourth neighboring checks of vf, f| and v| be cf, and c|, respectively. Let = {cl,cl,cl}. 
In the second iteration: 



UJ2{V,C) 



1 ifve {^^1,^2}' c ^ 4 

1 if V eV^, ceC^ 

otherwise 



(22) 



UJ2(C,V 



For all V G V , uj2{:, v 



1 if c G {c\, cl, 4, 4, 4, 4}, w G 
1 if ceC^, V ^V^ 
otherwise 

{1}. We now have the following lemma: 



(23) 



{0}. For no G V^, uj2{:, v 



Lemma 8: There exists no variable v ^ {JV"^ such that lj2{-,v) = {1}. 

Proof: The proof is by contradiction. Let v ^ V^^ |J such that lJ2{-,v) = {!}. Then, v is 
connected to four checks in {cj, 4, C3, Cg, Cg, c^} |J C^. Note that only two neighbors of v can belong 
to {c\, cl, cl, cl, cl, c^} without introducing a four-cycle. This combined with the fact that there are at 
most three variable nodes in V"^ implies that there are only two cases: 

(a) V has two neighbors in {c\, cl, cl, cl, cl, c\} and two neighbors in C^, say cf and c|. In this case, 
the set of variable nodes [J{vl,V2,v} has 13 check nodes, violating the 6 ^ 14 condition. 



(b) V has one neighbor in {cj,c 



1 ^1 u ^1 u 



.cl,Cj} and three neighbors in C^. In this case, the set of 



2! C3, C5, 

variable nodes |J V'^ [J{v} has 14 check nodes, violating the 7^16 condition. ■ 
Hence, if a decision is made after the second iteration, the decoder is successful. 

Subgraph 3: The variables in induce the subgraph shown in Fig. |4(c)[ At the end of the first 
iteration, v}, vl and ^3 receive correct messages from all their neighboring check nodes. Moreover, 
there exists no variable which receives four incorrect messages from checks in the set C^. Hence, if a 
decision is made after the first iteration, the decoder is successful. 

Subgraph 4: The variables in induce the subgraph shown in Fig. |4(d)[ At the end of the first 
iteration: 

1 if ceC^\{cl,cl}, v^V^ 
oJi{c,v) = < 1 if ce {cl,cl}, V eV^ (24) 
otherwise 

For no f G V\V^, iJi{:, v) = {1} as it this would introduce a four-cycle. For any v G V"\V"^, uj2iv, c) = 1 
only if uJi{: \c, v) = {1}. This implies that v has three checks in the set C^\{cl, c}}. Let V'^ be the set 
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UJ2{V,C) 



(25) 



UJ2{C,V) 



(26) 



of such variables. We now have the following lemma: 

Lemma 9: There can be at most two variables in V'^. Moreover, there exists no check c e C\C^ 
which is shared by two variables in the set V'^. 

Proof: Let e V'^. Then, the set U{^i? ^i' ^3} most 13 checks which violates 

the 6 ^ 14 condition. Hence, V'^ has at most two variables. 

Next, let two variables vf, e V"^ share a check c e C\C^. Then, the set U{^i' ^2} most 
11 checks which violates the 5 —> 12 condition. ■ 
Let be the set of checks C\C^ which are connected to variables in V'^. In the second iteration we 
have: 

1 if t' e{v\,vl}, c ^ {c\,c\] 
1 if V — vl 

1 if V eV^, ceC^ 

otherwise 

1 if c G {ci4}, V ^ v\ 
1 if CG 4), v^V^ 
1 if ceC^, V ^V^ 
otherwise 

For no v G V\V^, uJ2i-,v) = {1}, for such a structure cannot exist without creating a four-cycle or 
violating one of 5 ^ 12 and 6 — 14 conditions. For any v G ^\(^^IJ^^)' ^3(^1 c) = 1 only if 
cJ2{- \c, v) = {!}. This implies that v has three neighbors in the set [j C^. Let be the set of such 
variables. We have the following lemma: 

Lemma 10: For the sets V"^ and V^, the following are true: 

(i) If = 2, then is empty. 

(ii) If \V^\ > 0, then l^^j ^ 1. 

(iii) \V^\<\. 

Proof: We prove the lemma part by part. 

(i) Suppose = 2 and that is not empty. Let G V^. Then, the set \J V"^ U{^i} is of size 6 
and has at most 13 checks which violates the 6 — > 14 condition. 

(ii) Suppose \V^\ > 0. Let v\ &V^. If V'^ is empty, then v\ is connected to three checks in C^. This is 
not possible as vl G V\ {V^ (J V^). Next, suppose that l^^l = 2. Let vl, vl G V^. Then V'^ \J{vl, v^, vf} 
has at most 13 checks which violates the 6 —> 14 condition. 
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(iii) Suppose vf,V2 G V^. By (ii), there exists a variable vf G V"^. Then, IJ{^i; ^i; ^2} has at most 
13 checks which violates the 6 ^ 14 condition. ■ 
Suppose = {vf}. Denote the fourth check of vf by cf. Then, we have at the beginning of the third 
iteration: 

1 if t; G {vlv^}, c i {4,4} 
u^[v, c)=\ 1 if t; G c G C2 (27) 
1 ii V = v\, c = c\ 

At the end of the fourth iteration, cJ3(:, v) = {0} for v G V^. Thus, if a decision is made at the end of 
this iteration, all f G V^^ are decoded correctly. Now we prove the following lemma: 

Lemma 11: There exists no t> G V\V^ such that ZJs(:,v) = {!}. 

Proof: Suppose that is empty and that there exists a variable v such that iJ3{:,v) = {!}. If 
V'^ is empty, then, v is connected to four checks in C^\{cl, C5, Cg, Cj}. This is not possible as it would 
cause a four-cycle. If V'^ is not empty, then v is connected to four checks in {C^\{c\, C5, Cg, Cj}) [j C*^. 
Then, we would have lJ2{:,v) = {!}. However, from above, no such variable exists. 

Next, suppose that = {vf} and that = {cf}. Then, cf is the only check such that ZJ2{c, :) = {0} 
and uJ3{c, :) = {1}. It follows then that for any v G V^\V'^, if ZJ3{:,v) = {!}, then v is connected to 
cf. Also, it is connected to three checks in the set {c\, cl, cl, cl, cl, c\q, c^}. Then the set of variables 
[J{vf, vf, v} has at most 12 checks. This violates the 6 ^ 14 condition. ■ 
Hence, if a decision is made after the third iteration, the decoder is successful. 

Subgraph 5: The variables in induce the subgraph shown in Fig. |4(e)[ For all v G V^, uji{v, :) = 
{1}, 1 < i < 3. There exist no v E V\V^ that receive three incorrect messages, for the existence of 
such a variable would violate the 4-^11 condition. Hence, for all v G V\V^, uji{v, :) = {0}, 1 < i < 3. 

Let V'^ be the set of variables that have two checks in the set C^\{c\, cl, Cj}. Let be the remaining 
two checks of these variables. At the beginning of the fourth iteration, the decoder switches to the 
Gallager-B mode. Then: 

1 ifve V\ ceC^\{c\,clc]} 
Lj^iv, c) = I 1 if V eV^, ceC^ (28) 

otherwise 

At the end of the fourth iteration, uJ4(:, v) = {0} for v G V^. Moreover, for no f G V\V^, uJii:, v) = {1}, 
as the existence of such a variable would either induce a four-cycle or violate one of the 5-^12,6^ 14, 
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7 ^ 16 or 8 — > 18 conditions (the arguments used are similar to the ones used for Subgraph 2 and 
Subgraph 4). Hence, if a decision is made at the end of the fourth iteration, the decoder is successful. 

Now we prove the necessity of the conditions of the theorem. We prove this by giving subgraphs 
which violate one condition and are not successfully decoded in four iterations. Since the validity of 
these claims can be checked easily, a detailed proof is omitted. 

Necessity of the 4^11 condition 

Consider the subgraph shown in Fig. [51 In this case, the 4-^11 condition is not satisfied and the 
errors are not corrected at the end of the fourth iteration. Hence, in order to guarantee the correction 
of three errors in four iterations, the 4 — > 11 condition must be satisfied. 




Fig. 5. A 4 10 subgraph 

Necessity of the 5 ^ 12 condition 

There exists no graph of girth six which satisfies the 4 ^ 11 condition but does not satisfy the 
5 ^ 12 condition. 

Necessity of the 6 — ^ 14 condition 

Consider the graph shown in Fig. [6l The graph shown satisfies the 4 — > 11 and the 5 ^ 12 conditions 
but not the 6 ^ 14 condition. The errors are not corrected in four iterations. Hence, in order to guarantee 
the correction of three errors in four iterations, the 6 ^ 14 condition must be satisfied. 

Necessity of the 7 ^ 16 condition 

Consider the graph shown in Fig. U\ The graph shown satisfies the probability 4^11, 5^12 and 
the 6 ^ 14 conditions but not the 7 ^ 16 condition. The errors are not corrected at the end of the 
fourth iteration. Hence, in order to guarantee the correction three errors in four iterations, the 7 — 16 
condition must be satisfied. 
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1^ ^^ 



Fig. 6. A 6 — > 13 subgraph 




Fig. 7. A 7 ^ 15 subgraph 

Necessity of the 8 ^ 18 condition 

Consider the graph shown in Fig. [U The graph shown satisfies the 4 ^ 11, 5 ^ 12, 6 ^ 14 and 
the 7 ^ 16 condition but not the 8 ^ 18 condition. The errors are not corrected at the end of the 
fourth iteration. Hence, in order to guarantee the correction of three errors in four iterations, the 8^18 
condition must be satisfied. ■ 

In this section, we proved necessary and sufficient conditions to guarantee the correction of three 
errors in column-weight-four codes using an iterative decoding algorithm. By analyzing the messages 
being passed in subsequent iterations, it may be possible to get smaller bounds on the number of check 
nodes required in the "small" subgraphs. However, we hypothesize that the size of subgraphs to be 
avoided would be larger. 
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Fig. 8. A 8 — > 17 subgraph 



V. Numerical Results 

In this section, we describe a technique to construct codes with column-weight three and four which 
can correct three errors. Codes capable of correcting a fixed number of errors show superior performance 
on the BSC at low values of transition probability a. This is because the slope of the FER curve is 
related to the minimum critical number [[T4l. A code which can correct i errors has minimum critical 
number at least i + 1 and the slope of the FER curve is i + 1. We restate the arguments from [[T4| to 
make this connection clear. 

Let a be the transition probability of a BSC and be the number of configurations of received bits 
for which k channel errors lead to codeword (frame) error. The frame error rate (FER) is given by: 

n 

FER{(x) = Cka\l - 

k=i 

where i is the minimal number of channel errors that can lead to a decoding error and n is length of 
the code. 

On a semi-log scale the FER is given by 

\og{FER{a)) = logl^^Cka'^il-aT-"^ 

= l0g(Q)+^l0g(«)+l0g((l-«)'^-') 

+ log f 1 + - a)-' + ... + - a)-'] 

\ Ci Ci J 

For small a, the expression above is dominated by the first two terms. That is, 

log {FER{a)) ^ log(cj) + i log(a) 
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The \og{FER) vs. log(a) graph is close to a straight line with slope equal to i, the minimal critical 
number. If two codes Ci and C2 have minimum critical numbers ii and i2, such that ii > i2, then the 
code Ci will perform better than C2 for small enough a, independent of the number of trapping sets. 

A. Column-Weight-Three Codes 

From the discussion in above and in Section |nil it is clear that for a code to have an FER curve 
with slope at least 4, the corresponding Tanner graph should not contain the trapping sets shown in 
Fig. [U as subgraphs. We now describe a method to construct such codes. The method can be seen as a 
modification of the PEG construction technique used by Hu et al. [fTTI . The algorithm is detailed below 
as Algorithm [U 

Algorithm 1: ConstructCode 
Data: The set of n variable nodes (V) and m check nodes (C). The column weight of the code (7) 

Result: Code with column weight 7 

for j = 1 to n do 

for /c = 1 to 7 do 

if = 1 then 

Connect the A;*'^ edge of variable node j to the check node with the smallest positive 

degree, 
else 

Expand the tree rooted at node j to a depth of 6. 

Assimilate all check nodes which do not appear in the tree into CjTp, the set of 

candidates for connecting variable node j to. 

while A;*^ edge is not found do 

Find the check node Cj in Cj^ with the lowest degree. If connecting Cj to variable 

node j does not create a (5, 3) trapping set, set this as the k^^ edge. If it does, 

remove Cj from Cjif. 

end 

end 

end 

end 



Note that checking for a graph isomorphic to (8, 0) trapping set is computationally complex. Since, the 
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Fig. 9. Performance comparison of the PEG code and the example code for column-weight three 



PEG construction empirically gives good codes, it is unlikely that it introduces a weight-eight codeword. 
However, once the graph is grown fully, it can be checked for the presence of weight-eight codewords 
and these can be removed by swapping few edges. 

Using the above algorithm, a column-weight-three code with 504 variable nodes and 252 check nodes 
was constructed. The code has slight irregularity in check degree. There is one check node degree five 
and one check node with degree seven, but the remaining have degree six. The code has rate 0.5. In 
the algorithm, we restrict maximum check degree to seven. The performance of the code on BSC is 
compared with the PEG code of same length. The PEG code is empirically the best known code at 
that length on AWGN channel 1151 . However, it has fourteen (5,3) trapping sets. Fig. |9] shows the 
performance comparison of the two codes. As can be seen, the new code performs better than the 
original PEG code at small values of a. 

B. Column-Weight-Four Codes 

Unlike column-weight-three codes, the construction of column-weight-four codes involves ensuring 
certain expansion on subsets of variable nodes. This can be done only in time which grows exponentially 
with the length of the code. Hence,we consider the 4 ^ 12 condition rather than the necessary and 
sufficient conditions discussed in Section |IVl It can be shown that the 4 — > 12 condition is sufficient 
for the 4 — 11, 5 — 12, 6 — > 14, 7 — >^ 16 and the 8 — > 18 conditions. There are only two graphs 
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of girth 6 with 4 variable nodes and 11 check nodes. Fig. \W\ shows these two graphs. Avoiding these 
two subgraphs will ensure a code which can correct three errors. An algorithm for the construction of 
such codes is similar to the modified PEG algorithm given in Algorithm [TJ This algorithm was used 
to generate a code of length 816, girth 6 and rate 0.5. The code constructed has a slight irregularity in 
that three check nodes have degree nine and three have degree seven. 

Remark: For the code parameters given above, it was possible to generate a code which satisfied the 
4 ^ 12 condition. However, it might not be possible to satisfy this condition for codes with higher rate 
and/or shorter lengths. Should such a scenario arise, the set of subgraphs to be avoided should be changed 
(e.g., to those specified in the necessary and sufficient conditions). However, the code construction time 
will be larger. Hence, at the cost of code-construction time and complexity, it is possible to achieve 
shorter lengths and/or higher rates. 




(a) (b) 

Fig. 10. Graphs with girth 6 which have 4 variable nodes and 11 check nodes. Subgraphs with 4 variable nodes and fewer than 11 
check nodes do not exist. 

Fig. El shows the performance of the code under message-passing decoding. The curve on the left 
corresponds to four iterations of message-passing. The curve in the right corresponds to 25 iterations 
of message-passing. 

After only four iterations, errors of weight four and above were encountered which were not corrected 
by the message-passing decoder. However, after 25 iterations, the smallest weight error pattern still 
remaining had a weight of 7. We note that the average slope of the FER curve is 8 which is the 
weight of the dominant error event at these probabilities of error. This suggests that analysis over a 
higher number of iterations and on "larger" subgraph search will yield a stronger result. However, this 
is beyond the scope of this paper. Also, it is worth noting that the conditions of Theorem |4] avoid 
codewords of length 4 through 8 which improves the minimum distance of the code. 
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Fig. 11. Performance of the example colurmi- weight-four code for different numbers of iterations of message-passing 



VI. Conclusion 

In this paper, we provided a method to derive conditions that guarantee the correction of a finite 
number of errors by hard-decision decoding. Although more involved than the expander arguments used 
in previous works, it results in better bounds. Moreover, in contrast to previous expansion arguments, 
our results give rise to code-construction techniques that yield codes with guaranteed error-correction 
ability under message-massing decoding at practically feasible lengths. This method can be applied to 
(a) provide conditions for guaranteed correction of a larger number of errors, (b) yield similar results 
for higher column- weights and/or higher girths. However, such applications would be more involved 
than the analysis done in this work. 
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